Panorama synthesis processing of a plurality of image data

ABSTRACT

This invention is an image processing apparatus for generating graphics data representing a single seamless planar image synthesized from a multiple sets of graphics data contained in a plurality of graphics files. Each of the plurality of graphics files contains graphics data composed of a multiplicity of planar pixels arrayed in a plane to create a planar image. This image processing apparatus establishes a spheroidal or cylindrical projection plane for synthesis of the multiple sets of graphics data.

TECHNICAL FIELD

The present invention relates to image processing technology for seamingtogether multiple data representing an image.

BACKGROUND ART

In recent years, it has become common practice to capture photos using adigital still camera (DSC), and to then store the graphics datarepresenting the photos on a computer. A process termed panoramasynthesis, in which data for multiple images is synthesized to producegraphics data representing a single seamless image is sometimes carriedout as well.

However, images generated by a digital still camera have somedistortion, particularly in peripheral edge portions. This distortioncan create the problem of degraded image quality in image areas that areseamed together when synthesizing multiple images. One cause of thisdistortion is that light entering through the optical system of thedigital camera is projected onto an image pickup device having a flatphotoreceptor surface.

DISCLOSURE OF THE INVENTION

With the foregoing in view, it is an object of the present invention toprovide technology that overcomes the aforementioned drawbacks of theconventional art, by controlling image degradation produced in theprocess of synthesizing data for multiple images generated using a flatimage pickup device.

In order to attain the above and the other objects of the presentinvention, there is provided the first configuration of image processingapparatus for generating graphics data representing a single seamlessplanar image synthesized from a multiple sets of graphics data containedin a plurality of graphics files, in response to the plurality ofgraphics files each of which contains the graphics data composed of amultiplicity of planar pixels arrayed in a plane for representing aplanar image. The image processing apparatus comprising a synthesis areaestablisher configured to establish a spheroidal projection planecentered on a predetermined point, as an area for synthesis of themultiple sets of graphics data; a spheroidal image generator configuredto generate a plurality of spheroidal images, by projecting each ofplanar images represented by each of the multiple sets of graphics dataonto the projection plane; a feature point extractor configured toextract a feature point which is an area having a predeterminedcharacteristic, from each of the plurality of spheroidal images; acorrespondence relationship determiner configured to determine acorrespondence relationship of the extracted feature points, between theplurality of spheroidal images; a spheroidal image synthesizerconfigured to generate seamless spheroidal graphics data representing asingle seamless spheroidal image, by synthesizing a plurality ofgraphics data each of which representing each of the spheroidal images,with reference to the determined correspondence relationship; and aplanar image generator configured to generate the graphics datarepresenting the single seamless planar image, from the seamlessspheroidal image graphics data.

According to this first configuration of the invention, a spheroidalprojection plane centered on a predetermined point is established as anarea for synthesizing the multiple graphics data, and planar imagesrepresenting data for each of the multiplicity of images are projectedonto the established projection plane in order to generate amultiplicity of spheroidal images, which spheroidal images are thensynthesized. According to this aspect, since images are synthesized on aspheroidal projection plane, it is possible to control degradation inimage quality produced in the process of synthesizing data for multipleimages generated using a flat image pickup device.

In the image processing apparatus of the present invention, theplurality of graphics files may further include image attributeinformation which is attribute information of the graphics data. Theimage processing apparatus may further comprise a focal distancedeterminer configured to determine a focal distance of an optical systemused to generate the multiple sets of graphics data for each of themultiple set of graphics data, in response to the image attributeinformation; and the spheroidal image generator generates the pluralityof spheroidal images by projecting each planar image represented by eachof the multiple sets of graphics data onto the projection plane, theeach planar images being placed at a location away from thepredetermined point to the projection plane side, by the focal distancecorresponding to each of the multiple sets of graphics data.

By so doing, it is possible to carry out panorama synthesis even wherethe data for a multiplicity of images has been created at differentmagnification by means of using the zoom function during shooting, forexample.

In the image processing apparatus of the present invention, the imageattribute information may include lens focal distance representing focaldistance of a shooting lens; focal plane resolution unit specifying anunit of resolution in a focal plane of the optical system; focal planeheight resolution representing a pixel count in a pixel height directionper the focal plane resolution unit; and focal plane width resolutionrepresenting a pixel count in a pixel width direction per the focalplane resolution unit; the focal distance determiner determines the lensfocal distance to be the focal distance; and the spheroidal imagegenerator determines a pixel size in a width direction by means ofdividing the focal plane resolution unit by focal plane widthresolution, and also determines a pixel size in a height direction bymeans of dividing the focal plane resolution unit by the focal planeheight resolution.

In the image processing apparatus of the present invention, the imageattribute information may include 35 mm-equivalent lens focal distancewhich is a value of focal distance converted to a 35 mm film camerabasis. The focal distance determiner may determine the 35 mm-equivalentlens focal distance to be the focal distance. The spheroidal imagegenerator may determine 35 mm film size as a size of the planar image.

In the preferred image processing apparatus of the present invention,the image attribute information includes focal plane resolution unitspecifying an unit of resolution in a focal plane of the optical system;focal plane height resolution representing a pixel count in a pixelheight direction per the focal plane resolution unit; and focal planewidth resolution representing a pixel count in a pixel width directionper the focal plane resolution unit. The spheroidal image generatorcomprises: a spheroidal pixel establisher configured to establishspheroidal pixels on the spheroidal projection plane, the spheroidalpixel being allocated in a height direction by an angle divided by alargest one of the determined focal distances and the focal plane heightresolution, and being allocated in a width direction by an angle dividedby the largest determined focal distance and the focal plane widthresolution; and a spheroidal pixel value determiner configured todetermine each pixel value for each of the spheroidal pixels, accordingto a pixel value of a planar pixel projected onto each of the spheroidalpixels.

In this way, by establishing spheroidal pixels on the basis of thelargest one among the determined focal point distances, it becomespossible to carry out panorama synthesis without reducing the amount ofinformation of planar images having the smallest pixels, and withoutincluding an excessive amount of information in the spheroidal imagesgenerated from the planar images.

The second configuration of the invention is an image processingapparatus for generating graphics data representing a single seamlessplanar image synthesized from a multiple sets of graphics data containedin a plurality of graphics files, in response to the plurality ofgraphics files each of which contains the graphics data composed of amultiplicity of planar pixels arrayed in a plane for representing aplanar image. The image processing apparatus comprises: a synthesis areaestablisher configured to establish a cylindrical projection planecentered on a predetermined axis, as an area for synthesis of themultiple sets of graphics data; a cylindrical image generator configuredto generate a plurality of cylindrical images, by projecting each ofplanar images represented by each of the multiple sets of graphics dataonto the projection plane; a feature point extractor configured toextract a feature point which is an area having a predeterminedcharacteristic, from each of the plurality of cylindrical images; acorrespondence relationship determiner configured to determine acorrespondence relationship of the extracted feature points, between theplurality of cylindrical images; a cylindrical image synthesizerconfigured to generate seamless cylindrical graphics data representing asingle seamless cylindrical image, by synthesizing a plurality ofgraphics data each of which representing each of the cylindrical images,with reference to the determined correspondence relationship; and aplanar image generator configured to generate the graphics datarepresenting the single seamless planar image, from the seamlesscylindrical image graphics data.

In the preferred image processing apparatus of the present invention,the cylindrical image generator establishes the axis parallel to theheight direction established in the graphics data.

By so doing, it is possible to reduce image distortion of in the lateraldirection of images during the panorama synthesis process, whichtypically involves seaming together in the lateral direction.

The technique of the present invention is actualized by a variety ofapplications, which include image processing methods, computer programsthat attain the functions of such apparatuses and methods, recordingmedia in which such computer programs are recorded, and data signalsthat include such computer programs and are embodied in carrier waves.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration showing an image processing system as anembodiment of the invention.

FIG. 2 is a block diagram showing a simplified arrangement of a digitalstill camera as the input apparatus for generating graphics data.

FIG. 3 is a block diagram showing a simplified arrangement of a computerPC and a color printer as output apparatus for outputting graphics data.

FIG. 4 is an illustration showing a simplified arrangement of graphicsfile GF structure in the embodiment of the invention.

FIG. 5 is an illustration showing an example of attribute informationstored in the Exif IFG of a graphics file GF.

FIG. 6 is a flowchart showing the processing routine of the panoramasynthesis process in a computer PC.

FIGS. 7(a) and 7(b) are illustrations depicting the positionalrelationship between planar RGD data and a cylindrical coordinatesystem.

FIG. 8 is an illustration showing the processing routine of thecoordinate conversion process in the first embodiment of the invention.

FIGS. 9(a) and 9(b) are illustrations depicting the positionalrelationship between planar RGB data and the projection plane CYL,viewed from the axis Caxis direction.

FIG. 10 is an illustration showing distortion of an image producedduring generation of graphics data using a planar image pickup device.

FIG. 11 is an illustration depicting the relationship between planar RGBdata and a spheroidal coordinate system.

FIGS. 12(a), 12(b), and 12(c) are illustrations depicting generation oftwo sets of cylindrical graphics data having the same focal distancefrom the scene.

FIGS. 13(a), 13(b), and 13(c) are illustrations depicting feature pointsextracted from each set of cylindrical graphics data.

FIG. 14 is an illustration showing a corresponding point search processbeing carried out.

FIGS. 15(a) and 15(b) are illustrations giving a description of thesynthesis process.

FIGS. 16(a), 16(b), and 16(c) are illustrations depicting graphics datarepresenting a single seamless image being generated from two sets ofgraphics data having different focal distances.

FIG. 17 is an illustration depicting generation of two sets ofcylindrical graphics data from two sets of graphics data havingdifferent focal distances.

BEST MODE FOR CARRYING OUT THE INVENTION

The following description of the best mode for carrying out theinvention on the basis of embodiments shall be made in the orderindicated below.

A. Arrangement of Image Processing System:

B. Arrangement of Image File:

C. Panorama Synthesis Process in the first embodiment:

D. Panorama Synthesis Process in the second embodiment:

E. Variations:

A. Arrangement of Image Processing System:

FIG. 1 illustrates an image processing system 10 in one embodiment ofthe invention. The image processing system 10 includes a digital stillcamera 12 functioning as an input device that generates original imagedata, a personal computer PC functioning as an image processing devicethat performs image processing of the original image data generated bythe digital still camera 12, and a color printer 20 functioning as anoutput device that outputs processed images.

The digital still camera 12, the personal computer PC, and the colorprinter 20 are mutually connectable via a cable CV. In the state ofconnection by the cable CV, the digital still camera 12 and the otherconstituents are allowed to transmit and receive image files via thecable CV. In the status of no connection by the cable CV, the digitalstill camera 12 and the other constituents are allowed to transmit imagefiles therebetween via a memory card MC.

FIG. 2 is a block diagram schematically illustrating the structure ofthe digital still camera 12 working as the input device of generatingimage data. The digital still camera 12 focuses an image through anoptical lens on a charge coupled device (CCD) so as to electricallyrecord a still image.

The digital still camera 12 includes an optical circuit 121 having a CCDthat converts optical signals into electrical signals, an imageacquisition circuit 122 that controls the optical circuit 121 to acquireimage data, an image processing circuit 123 that processes the acquiredimage data, and a control circuit 124 that controls these respectivecircuits. The digital still camera 12 further has a selection/decisionbutton 126 as a user interface and a liquid crystal display 127 used forpreview of photographed images and user interfaces.

The shooting process (graphics data capture process) by the digitalstill camera 12 is carried out in the order: (1) setting of shootingmode by the user; (2) shooting (input of graphics data); (3) imageprocessing; and (4) recording of graphics file. Shooting mode settingincludes setting of lens focal distance. Setting of lens focal distanceis carried out by exchanging lenses (not shown) or manipulating a zoomlens (not shown).

The photographing is performed by pressing a shutter button by the user.In response to the press of the shutter button, the photographing isperformed with a focal distance available by the lens equipped with thedigital still camera 12. For example, when photographed with a zoom lensproviding a variable focal distance in response to the user operation,the photographing and the original image data generation are performedwith an adjusted focal distance.

The original image data thus generated are subjected to image processingfor storage. This image processing is a pre-treatment of the originalimage data, prior to storage into the memory card MC. The generalprocedure converts the original image data into a JPEG format suitablefor storage of photographic images. After conversion into the JPEGformat, the procedure adds photographing information PI to the convertedimage data to create an image file.

The photographing information PI regards photographing conditions andincludes information representing the selected light metering. Theprocess of acquiring image data in the digital still camera 12terminates with recording of the image file into the memory card MC. Thestructure of the image file will be discussed later.

FIG. 3 is a block diagram schematically showing the configuration of thecomputer PC and the color printer 20 as the output device of outputtingimage data. The computer PC has a slot 22 for reading an image file fromthe memory card MC inserted therein and a print data generation circuit23 for generating print data, according to which the color printer 20carries out printing. The print data generation circuit 23 includes acentral processing unit (CPU) 231 that executes arithmetic operationsfor generation of print data, a hard disk 232 that stores programsexecuted by the CPU 231, results of the arithmetic operations by the CPU231, and other data, and a random access memory (RAM) 233 thattemporarily stores these programs and data.

The color printer 20 is capable of outputting color images. One typicalexample of the color printer 20 is an ink jet printer that ejects fourdifferent color inks, cyan (C), magenta (M), yellow (Y), and black (K)on a printing medium to form a dot pattern and thereby complete aprinted image.

B. Arrangement of Image File:

FIG. 4 schematically shows the structure of an image file GF in theembodiment of the invention. The image file GF has a file structure inconformity with Exchangeable image file format (Exif) for digital stillcameras. This format is specified by Japan Electronics and InformationTechnology Industries Association (JEITA). According to this format,JPEG-Exif files that store compressed JPEG data as image data areincluded in Exif files (files of the Exif format).

The image file GF includes an SOI marker segment 101 representing aheader of compressed data, an APP1 marker segment 102 storing attributeinformation of the Exif format, an APP2 marker segment 103 storing Exifextended data, a DQT marker segment 104 defining a quantization table, aDHT marker segment 105 defining a Huffman table, a DRI marker segment106 defining an insertion interval of a restart marker, an SOF markersegment 107 representing various parameters relating to frames, an SOSmarker segment 108 representing various parameters relating to scanning,an EOI marker segment 109 representing a termination of the compresseddata, and an image data storage area 110.

The APP1 marker segment 102 stores an APP1 marker 1021, an Exifidentification code 1022, a TIFF header and other attribute information1023, and thumbnail image data 1024. The attribute information 1023 hasa TIFF structure with a file header (TIFF header) and includes, in thecase of an Exif-JPEG file, 0^(th) IFD storing attribute informationrelating to compressed image data, Exif IFD storing the photographinginformation PI and other attribute information inherent to the Exifformat and 1^(st) IFD storing attribute information relating tothumbnail images. The Exif IFD is pointed by an offset from the TIFFheader stored in the 0^(th) IFD. Tags for identifying various pieces ofinformation are used in the IFD, and the respective pieces ofinformation may be referred to as tag names.

FIG. 5 is an illustration showing an example of attribute informationstored in the Exif IFG of a graphics file GF. The attribute informationincludes tags of various kinds, including a tag relating to version andtags relating to shooting conditions. Tags relating to shootingconditions store exposure time, lens F stop, ISO sensitivity, shutterspeed, aperture, luminance, lens focal distance, focal plane widthresolution, focal plane height resolution, focal plane resolution unit,35 mm-equivalent lens focal distance, and other parameters, stored asshooting information PI in accordance with predetermined offset.Recording of the shooting information PI is carried out during shootingby the digital still camera 12, as described previously.

C. Panorama Synthesis Process in the First Embodiment:

FIG. 6 is a flowchart showing the processing routine of the panoramasynthesis process in a computer PC. In Step S100, the CPU 231 reads thegraphics file GF from the memory card MC which has been inserted intothe slot 22, and stores this in the RAM 233 belonging to the print datagenerating circuit 23. The graphics file GF stores graphics data in theJPEG file format as the graphics data GD. Graphics data in the JPEG fileformat is composed of compressed YCbCr data.

In Step S110, the CPU 231 performs a color conversion process afterhaving expanded the compressed YCbCr data. By means of this colorconversion process, the YCbCr data is converted to RGB data. The reasonfor conversion to RGB data is that RGB data is used in image processingin the personal computer PC and in the color printer 20. Both the RGBand the YCbCr data take the form of planar images composed of aplurality of pixels in a plane (herein also termed planar pixels)arrayed on a plane.

In Step S120, the CPU 231 performs a coordinate conversion process onthe RGB data. This coordinate conversion process is specifically aprocess in which the RGB data, which is a planar image, is projectedonto a cylindrical surface positioned in a cylindrical coordinatesystem, to generate graphics data on a cylindrical surface. Hereinafter,graphics data generated in this way shall be referred to as cylindricalRGB data, and RGB data composed as a planar image shall be referred toas planar RGB data.

FIGS. 7(a) and 7(b) are illustrations depicting the positionalrelationship between planar RGD data MI and a projection plane CYL in acylindrical coordinate system. FIG. 7(a) depicts the positionalrelationship between the optical system and the image pickup device ofthe digital still camera 12. The optical system is shown as a lens L.The image pickup device is composed as a CCD (Charge-Coupled Apparatus)having a flat photoreceptor surface.

The lens L replaces the several lenses of the digital still camera 12optical system with a single lens having the same effect. The center ofthe lens L is termed the principal point, and the plane passing throughthe principal point and perpendicular to the optical axis Laxis istermed the principal plane. As regards the lens L, there are defined anobjective surface indicating a subject, and an imaging surface whichfocuses light from the subject. The objective surface and the imagingsurface are defined as being situated spaced apart from the principalplane in the optical direction, by an objective surface distance a andan imaging surface distance b, respectively.

The image pickup device is positioned with its photoreceptor surfacealigned with this imaging surface. By means of this arrangement, lightfrom the subject on the objective surface passes through the lens L andis focused on the photoreceptor surface of the image pickup device.

FIG. 7(b) depicts the planar RGB data MI being projected onto theprojection plane CYL. Since the planar RGB data MI is graphics data thathas been generated using the image pickup device, pixel values of theplanar pixels PXL thereof are values that originally were generateddepending on light incident on the image pickup pixels disposed on theimaging surface.

The projection plane CYL is established as part of a cylindrical surfacewhich is a collection of a plurality of points situated away by imagingsurface distance b from the axis Caxis of the cylindrical coordinatesystem. The axis Caxis is an axis that passes through a point PPcorresponding to the principal point of FIG. 7(a) and extends in thevertical direction of the planar RGB data MI. It will be understood thatthe imaging surface distance h substantially corresponds to lens focaldistance. The value stored in the Exif IFD of the graphics file GF (FIG.5) can be used as lens focal distance. By so doing, a cylindrical imagehaving a plurality of cylindrical pixels defined by the size Z₀ in theCaxis direction and the angle θ₀ can be constructed on the projectionplane CYL.

FIG. 8 is an illustration showing the processing routine of thecoordinate conversion process in the first embodiment of the invention.In Step S210, the CPU 231 reads out from the graphics file GF data foruse in the coordinate conversion process. Data for use in the coordinateconversion process includes lens focal distance, focal plane resolutionunit, resolution of focal plane width, and resolution of focal planeheight. Lens focal distance refers to the actual focal distance of theshooting lens; as noted, it substantially corresponds to the distancefrom the center of a hypothetic single convex lens to the image pickupdevice where the light is focused.

Focal plane resolution unit specifies the unit for measuring resolutionof the width of the focal plane and resolution of the height of thefocal plane. Resolution of focal plane width represents the number ofpixels in the image width direction per unit of focal plane resolution.Resolution of focal plane height represents the number of pixels in theimage height direction per unit of focal plane resolution. Here, focalplane refers to the imaging plane in which light from the subjectfocuses; as noted, it is aligned with the photoreceptor surface of theimage pickup device.

In Step S220, the CPU 231 performs the following processes.

(1) The cylindrical coordinate system shown in FIG. 7(b) is defined.

(2) A cylindrical surface centered on the axis Caxis of the cylindricalcoordinate system is established on the cylindrical coordinate system.In this embodiment, to facilitate understanding, lens focal distance isdeemed the constant distance mentioned previously.

(3) Planar RGB data is placed at a predetermined location on thecylindrical coordinate system.

Placement of the planar RGB data is carried out at a location situatedaway, by the equivalent of the lens focal distance, from the axis Caxis,and with an orientation such that the centerline CL extending in theheight direction of the planar RGB data parallel with the axis Caxis.The reason for placing the data so that the centerline CL extending inthe height direction of the planar RGB data parallel with the axis Caxisis in order to control distortion of the image in the image lateraldirection in the panorama synthesis process, which typically involvesseaming together images in the lateral direction.

In Step S230, the CPU 231 determines the size of the planar pixels ofthe planar RGB data, and the size of the cylindrical pixels on theprojection plane CYL. With regard to the planar pixel size,specifically, pixel size in the width direction can be determined bydividing the focal plane resolution unit by the resolution of focalplane width, and pixel size in the height direction by dividing thefocal plane resolution unit by the resolution of focal plane height. Theplanar pixel size determined in this way will correspond to image pickupdevice pixel size, in the event that there no resampling of dataobtained from the image pickup device.

FIGS. 9(a) and 9(b) are illustrations depicting the positionalrelationship between planar RGB data and the projection plane CYL,viewed from the axis Caxis direction. FIG. 9(a) shows FIG. 7(b) viewedfrom above. FIG. 9(b) is a partial enlarged view of FIG. 9(a). In FIG.9(a), a plurality of planar pixel rows C are shown in the planar RGBdata MI. A planar pixel row C is a pixel row composed of a plurality ofplanar pixels lined up in the vertical direction in the planar RGB dataMI. In FIG. 9(a) there are shown 11 planar pixel rows, from planar pixelrow C5L to planar pixel row C5R. The planar pixels contained in eachplanar pixel row all have the same height Z₀ (in the axis Caxisdirection).

In FIG. 9(a) there is also shown a plurality of cylindrical pixel rows Pwithin the projection plane CYL. A cylindrical pixel row P is a pixelrow composed of a plurality of planar pixels lined up in the heightdirection in the projection plane CYL. In FIG. 9(a) there are shown 9cylindrical pixel rows, from cylindrical pixel row P4L to cylindricalpixel row P4R. All of the cylindrical pixel rows have width of the samegiven angle θ₀. The cylindrical pixels contained in each cylindricalpixel row all have the same height Z₀ (not shown).

The angle θ₀ representing cylindrical pixel width is established on thebasis of a planar pixel PXL in proximity to the optical axis Laxis.Specifically, a cylindrical pixel row P0 is determined by projecting aplanar pixel C0 onto the projection plane CYL, and the angle θ₀ of thecylindrical pixel row P0 so determined is designated as the angle of thecylindrical pixel rows. By establishing cylindrical pixels (not shown)in this way, it is possible to construct a cylindrical image in theprojection plane CYL (Step S240).

In Step S250, the CPU 231 extracts the pixel values of the cylindricalpixels, from the pixels values of the planar pixels of the planar RGBdata MI. FIG. 9(b) is a partial enlarged view of FIG. 9(a), illustratingthe method for extracting pixel values of cylindrical pixels using abilinear interpolation method. In the extraction example hereinbelow, inorder to facilitate the description, the RGB data is assumed to beone-dimensional data having only one row of planar pixels in the heightdirection; however, it could easily be expanded to two-dimensional databy carrying out the same process in the height direction.

In FIG. 9(b) are shown a cylindrical pixel P3R, the centerline P3RCL ofthe cylindrical pixel P3R, and the projection point X1 of the centerlineP3RCL onto the planar RGB data MI. The centerline P3RCL is a lineextending from the axis Caxis (FIG. 9(a)) towards the center of thecylindrical pixel P3R.

The pixel value of the cylindrical pixel P3R can be extracted in thefollowing manner, from the pixel values of the planar pixels C3R, C4R.The projection point X1 is projected into the planar pixel C3R, and issituated a distance 50 a away from the center location of the planarpixel C3R, and a distance 50 b away from the center location of theplanar pixel C4R. These distances 50 a, 50 b can be calculated from theplanar RGB data and the lens focal distance by the CPU 231.

Where a bilinear interpolation method is used, the pixel value of thecylindrical pixel P3R can readily be extracted with the calculationequation: ((pixel value of planar pixel C3R×distance 50 b+pixel value ofplanar pixel C4R×distance 50 a)/(distance 50 a+distance 50 b)). Othercalculation methods include the cubic convolution method and otherextraction methods. The reason for projecting the planar RGB data MIonto a cylindrical surface prior to synthesis of the graphics data is asfollows.

FIG. 10 is an illustration showing distortion of an image producedduring generation of graphics data using a planar image pickup device.In the drawing, there are shown two sets of planar RGB data M1, M2generated by projecting a given subject m from the same point, but atseparate angles. The two sets of planar RGB data M1, M2 are projectedrespectively on two optical axes Laxis1, Laxis2. In the planar RGB dataM1, the subject m is positioned in the center portion of the image,whereas in the planar RGB data M1, the subject m is positioned at theleft edge of the image.

As will be apparent from FIG. 10, in the planar RGB data M1 the subjectm has apparent size substantially the same as the planar pixel, whereasin the planar RGB data M2 it has apparent size larger than the planarpixel. In this way, the apparent size of the subject m in the planar RGBdata will differ depending on the location at which the subject m ispositioned in the image. As a result, image distortion occurs.

On the other hand, as will be apparent from FIG. 10, in a cylindricalimage generated on a cylindrical surface, the apparent size of thesubject m will be the same regardless of the location of the subject mon the image. In this way, by means of projecting a planar image onto acylindrical projection plane to effect conversion to a cylindricalimage, it is possible to minimize distortion in the lateral direction.By projecting onto a spheroidal coordinate system in the manner shown inFIG. 11, it becomes possible to minimize distortion in both the heightdirection and the width direction. Such an arrangement is markedlyeffective in cases where a multiplicity of images are being synthesizedin the height direction.

FIG. 12 illustrates generation of two sets of cylindrical graphics datahaving the same focal distance from a scene View. The two sets ofcylindrical graphics data are data generated by means of shooting thetwo frames Fa1, Fb1 shown in FIG. 12(a). The graphics data shown in FIG.12(b) has been derived by developing data shot in frame Fa1 andconverted to a cylindrical image; the graphics data shown in FIG. 12(c)has been derived by developing data shot in frame Fb1 and converted to acylindrical image.

In Step S130, the CPU 231 carries out a feature point extraction processon the two sets of cylindrical image data. The feature point extractionprocess is a process for extracting feature points that well representan external characteristic of a subject in an image. FIGS. 13(a), 13(b),and 13(c) are illustrations depicting feature points extracted fromeach, set of cylindrical graphics data. Each feature point need not havesize equivalent to one pixel, but may be an area composed of severalpixels.

The feature point extraction process may be carried out by the followingmethod, for example. First, the CPU 231 extracts the contour line as acollection of points, using a Sobel filter or other contour lineextraction filter. FIG. 13(a) shows contour line data PICTa1 generatedby performing a contour line extraction process on cylindrical imagedata PICTa0 (FIG. 12(b)).

Next, the CPU 231, using an SRA (Side effect resampling Algorithm) orother resampling algorithm, extracts a feature point from the collectionof points making up the extracted contour line. FIG. 13(b) shows featurepoint data PICTa2 generated by performing a feature point extractionprocess on contour line data PICT1. FIG. 13(c) shows feature point dataPICTb2 generated by performing a feature point extraction process oncontour line data PICTb1 (not shown).

The feature point data PICTa2 includes two feature points C1, Ca2, andthe feature point data PICTb2 includes two feature points Cb1, Cb2. InFIG. 13(b) and FIGB. 13(c), contour lines are shown overlapped tofacilitate understanding.

In Step S140, the CPU 231 performs a corresponding point search processon two sets of feature point data PICTa2, PICTb2. The correspondingpoint search process is a process that searches for features points thatcorrespond to one another among multiple images. This process is aprocess for determining a given location in a given subject amongmultiple sets of graphics data targeted for panorama synthesis.

The corresponding point search process can be carried out by means ofsearching for a collection of a plurality of feature points that meetthe following criteria, for example.

-   (1) Pixels surrounding a feature point (e.g. 3×3 pixels) have pixel    value differences within a predetermined threshold value. Feature    points associated in this way are termed approximate feature points.-   (2) In each set of feature point data, a plurality of approximate    feature points have the same positional relationship. Approximate    feature points associated in this way are determined to be    corresponding points.

FIG. 14 is an illustration showing the corresponding point searchprocess being carried out. The feature point data PICTa2 includes twofeature points Ca1, Ca2, and the feature point data PICTb2 includes twofeature points Cb1, Cb2. Feature point Ca1 and feature point Cb1represent the same subject, and thus the surrounding pixels haveapproximate pixel values. As a result, feature point Ca1 and featurepoint Ca2 will be found to be corresponding approximate feature points.Similarly, feature point Ca2 and feature point Cb2 will be found to becorresponding approximate feature points.

In feature point data PICTa2, there is recognized a positionalrelationship whereby approximate feature point Ca2 is positioned to theupper right of approximate feature point Ca1. On the other hand, infeature point data PICTb2, approximate feature point Cb2, whichapproximates approximate feature point Ca2, is positioned to the upperright of approximate feature point Cb1, which approximates approximatefeature point Ca1. In this way, it is ascertained that the in each setof feature point data PICTa2, PICTb2, a plurality of approximate featurepoints have the same positional relationship. It can therefore bedetermined that feature point Ca1 of feature point data PICTa2corresponds to feature point Cb1 of feature point data PICTb2, and thatfeature point Ca2 of feature point data PICTa2 corresponds to featurepoint Cb2 of feature point data PICTb2.

In Step S150, the CPU 231 performs a synthesis process of two sets ofcylindrical graphics data. The synthesis process is a process forsynthesizing multiple sets of graphics data in such a way thatcorresponding feature points align.

FIGS. 15(a) and 15(b) are illustrations giving a description of thesynthesis process. The synthesis process is carried out by determiningthe positional relationship of two images in such a way thatcorresponding feature points are positioned in proximity, and thesubjecting each image to localized deformation so that positions ofcorresponding feature points align.

FIG. 15(a) shows two sets of feature point data PICTa2, PICTb2 arrangedso that corresponding feature points are positioned in proximity. Inthis example, feature point Ca1 belonging to feature point data PICTa2is located just to the right of feature point Cb1 belonging to featurepoint data PICTb2. Feature point Ca2 belonging to feature point dataPICTa2 is located just to the left of feature point Cb2 belonging tofeature point data PICTb2.

In this embodiment, this placement is due to the fact that the distancebetween feature point Ca1 and feature point Ca2 is smaller than thedistance between feature point Cb1 and Cb2, due to distortion of theimages. The presence of distortion in each image is due to the factthat, while the image distortion shown in FIG. 10 has been suppressed,there is residual distortion due to lens aberration. Where it is assumedthat each image will have some distortion in this way, it is preferableto determine relative positions of the two images so as to minimize thesquares of the distances between corresponding points, for example.

Next, positional relationships of corresponding points are generated asvector data. This vector data is used during localized affine conversionof graphics data such that corresponding points align. One cylindricalimage synthesized in this way is depicted in FIG. 15(b). By “unrolling”this cylindrical image in the lateral direction, a seamless planar imagecan be generated (Step S160).

In this way, in this embodiment, by synthesizing an image in acylindrical coordinate system that reproduces the positionalrelationship between the optical system and the image pickup device whengraphics data is generated in the digital still camera 12, since imagedistortion in the lateral direction of the image occurring when an imageis generated using a planar image pickup device at a given focaldistance is reduced, it is possible to suppress degradation in imagequality produced during the process of synthesizing multiple sets ofgraphics data created using the image pickup device.

The functions of the focal distance determiner, the synthesis areaestablisher, the spheroidal image generator, the feature pointextractor, the correspondence relationship determiner, the spheroidalimage synthesizer, and the planar image generator recited in the claimsare carried out by the CPU 231.

D. Panorama Synthesis Process in the Second Embodiment:

FIGS. 16(a), 16(b), and 16(c) are illustrations depicting graphics datarepresenting a single seamless image being generated from two sets ofgraphics data having different focal distances. The two sets of graphicsdata are the graphics data generated by means of shooting the two framesFf1, Fn1 shown in FIG. 16(a). The reason that the two frames Ff1, Fn1differ in size (angle of field) is that the focal distance has beenchanged by means of a zoom operation or changing lenses. Specifically,the graphics data of frame Ff1 has been generated at a relatively longfocal distance, and the graphics data of frame Fn1 has been generated ata relatively short focal distance.

FIG. 16(b) shows one graphics data frame F3 generated from two sets ofgraphics data. The size of frame F3 has image width generated bysynthesizing the two frames Ff1, Fn1, with the height of the frame Ff1which, of the two frames Ff1, Fn1, has the longer focal distance(smaller angle of field) being trimmed. FIG. 16(c) shows a seamlessplanar image PICT2 generated to have the frame F3 determined in thisway.

FIG. 17 is an illustration depicting generation of two sets ofcylindrical graphics data from two sets of graphics data havingdifferent focal distances. Planar RGB data MIn is data generated byshooting the frame Fn1. Planar RGB data MIf is data generated byshooting the frame Ff1 (FIG. 16(a)). The planar RGB data MIn and theplanar RGB data MIf are positioned away from the cylindrical coordinatesystem axis Caxis in the optical axis Laxis direction, by a focaldistance Rn and a focal distance Rf, respectively. The focal distance Rnand focal distance Rf are values for lens focal distance read from theExif IFD of each graphics file GF.

The projection plane CYL on the cylindrical coordinate system is aprojection plane established with the cylindrical coordinate system axisCaxis as its center, and having a radius equal to the focal distance Rf.The focal distance Rf is the longer of the two different focal distancesestablished by shooting the two sets of planar RGB data MIn, MIf. Theangle θ₀ representing the cylindrical pixel width is established by amethod similar to that of the first embodiment (FIG. 9(a)), on the basisof the planar pixels (not shown) of the planar RGB data MIf.

The reason that the cylindrical pixel angle θ₀ is established on thebasis of the planar pixels of the planar RGB data MIf is that if it werebased on the planar RGB data MIn, the cylindrical pixel angle θ₀ wouldbe excessive with respect to the planar RGB data MIf, and informationbelonging to the planar RGB data MIf would be lost. On the other hand,if the cylindrical pixel angle θ₀ were made smaller than this, theamount of data of the cylindrical image would become excessive withrespect to information belonging to the planar RGB data MIf, MIn.

The pixel value of the cylindrical pixel P4L can be calculated asfollows from pixel values of planar pixels of the planar RGB data MIf,MIn, in the same manner as in the first embodiment. The cylindricalimage pixel P4L generated by converting the planar RGB data MIf iscalculated from two pixels C4Lf, C5Lf of the planar RGB data MIf.Meanwhile, the cylindrical image pixel P4L generated by converting theplanar RGB data MIn is calculated from two pixels C2Ln, C3Ln of theplanar RGB data MIn.

In this way, the invention is applicable to cases in which a panoramasynthesis process is carried out on two sets of graphics data havingdifferent focal distances.

E. Variations

The invention is not limited to the embodiments and embodiments setforth hereinabove, and may be reduced to practice in various other modeswithout departing from the spirit thereof, such as the followingvariations, for example.

-   E-1. Whereas in the embodiments hereinabove, a single seamless    planar image is generated by unrolling cylindrical image data that    has been arranged on a projection plane having a cylindrical shape,    a single seamless planar image could instead be generated by means    of projecting onto a plane graphics data that has been composed on a    projection plane having a cylindrical shape or a spheroidal shape.    Generally, the planar image generator used in the present invention    may be any arrangement that generates graphics data representing a    seamless planar image, from graphics data such as spheroidal    graphics data or cylindrical graphics data.-   E-2. Whereas in the embodiments hereinabove, coordinate conversion    to a cylindrical coordinate system employs lens focal distance    stored in the EXIF IFD of each graphics file GF, it would instead be    possible to effect coordinate conversion to a cylindrical coordinate    system using 35 mm-equivalent lens focal distance stored in the EXIF    IFD of each graphics file GF, for example.

In this case, planar pixel size of planar RGB data can be determineddepending on 35 mm film size and pixel count. Specifically, pixel heightcan be calculated by dividing the length in the height direction of 35mm film size by the pixel count, and pixel width can be calculated bydividing the length in the width direction of 35 mm film size by thepixel count.

-   E-3. Whereas in the embodiments hereinabove, the panorama synthesis    process is carried out after the YCbCr data has been converted to    RGB data, it would instead be possible to carry out the panorama    synthesis process is carried out before the YCbCr data has been    converted to RGB data. In this latter case, there could be employed    an arrangement in which the feature point extraction and the    corresponding point search are carried out exclusively on the basis    of luminance information to which the human eye is highly sensitive,    for example. By so doing, a resultant advantage is the ability to    achieve a panorama synthesis process with a minimal amount of    calculation, without excessive degradation of image quality.-   E-4. Whereas in the embodiments hereinabove, a personal computer    functions as the image processing apparatus, it would instead by    possible for the color printer or the digital still camera to have    the function of an image processing apparatus. Also, the invention    is not limited to color printing only, but is applicable to    monochrome printing as well.-   E-5. Whereas in the embodiments hereinabove, an ink-jet color    printer is used as the output apparatus, the invention is applicable    also in instances where a monitor such as a CRT display or LCD    display, a projector, or some other apparatus able to display images    is used as an output apparatus.

Where some or all of the functions of the present invention are realizedthrough software, the software (computer program) may be provided in aform stored on a computer-readable recording medium. In the presentinvention, “computer-readable recording medium” is not limited toportable recording media such as flexible disks and CD-ROM, but includesalso various flavors of RAM, ROM, and other computer internal storageapparatus, as well as hard disks and other external storage apparatusfixed in a computer.

INDUSTRIAL APPLICABILITY

This invention is applicable to computer output apparatus.

1. An image processing apparatus for generating graphics datarepresenting a single seamless planar image synthesized from a multiplesets of graphics data contained in a plurality of graphics files, inresponse to the plurality of graphics files each of which contains thegraphics data composed of a multiplicity of planar pixels arrayed in aplane for representing a planar image, the image processing apparatuscomprising: a synthesis area establisher configured to establish aspheroidal projection plane centered on a predetermined point, as anarea for synthesis of the multiple sets of graphics data; a spheroidalimage generator configured to generate a plurality of spheroidal images,by projecting each of planar images represented by each of the multiplesets of graphics data onto the projection plane; a feature pointextractor configured to extract a feature point which is an area havinga predetermined characteristic, from each of the plurality of spheroidalimages; a correspondence relationship determiner configured to determinea correspondence relationship of the extracted feature points, betweenthe plurality of spheroidal images; a spheroidal image synthesizerconfigured to generate seamless spheroidal graphics data representing asingle seamless spheroidal image, by synthesizing a plurality ofgraphics data each of which representing each of the spheroidal images,with reference to the determined correspondence relationship; and aplanar image generator configured to generate the graphics datarepresenting the single seamless planar image, from the seamlessspheroidal image graphics data.
 2. The image processing apparatus inaccordance with claim 1, wherein the plurality of graphics files furtherinclude image attribute information which is attribute information ofthe graphics data, wherein the image processing apparatus furthercomprises a focal distance determiner configured to determine a focaldistance of an optical system used to generate the multiple sets ofgraphics data for each of the multiple set of graphics data, in responseto the image attribute information; and the spheroidal image generatorgenerates the plurality of spheroidal images by projecting each planarimage represented by each of the multiple sets of graphics data onto theprojection plane, the each planar images being placed at a location awayfrom the predetermined point to the projection plane side, by the focaldistance corresponding to each of the multiple sets of graphics data. 3.The image processing apparatus in accordance with claim 2, wherein theimage attribute information includes lens focal distance representingfocal distance of a shooting lens; focal plane resolution unitspecifying an unit of resolution in a focal plane of the optical system;focal plane height resolution representing a pixel count in a pixelheight direction per the focal plane resolution unit; and focal planewidth resolution representing a pixel count in a pixel width directionper the focal plane resolution unit; the focal distance determinerdetermines the lens focal distance to be the focal distance; and thespheroidal image generator determines a pixel size in a width directionby means of dividing the focal plane resolution unit by focal planewidth resolution, and also determines a pixel size in a height directionby means of dividing the focal plane resolution unit by the focal planeheight resolution.
 4. The image processing apparatus in accordance withclaim 2, wherein the image attribute information includes 35mm-equivalent lens focal distance which is a value of focal distanceconverted to a 35 mm film camera basis; the focal distance determinerdetermines the 35 mm-equivalent lens focal distance to be the focaldistance; and the spheroidal image generator determines 35 mm film sizeas a size of the planar image.
 5. The image processing apparatus inaccordance with claim 2, wherein the image attribute informationincludes focal plane resolution unit specifying an unit of resolution ina focal plane of the optical system; focal plane height resolutionrepresenting a pixel count in a pixel height direction per the focalplane resolution unit; and focal plane width resolution representing apixel count in a pixel width direction per the focal plane resolutionunit; the spheroidal image generator comprises: a spheroidal pixelestablisher configured to establish spheroidal pixels on the spheroidalprojection plane, the spheroidal pixel being allocated in a heightdirection by an angle divided by a largest one of the determined focaldistances and the focal plane height resolution, and also beingallocated in a width direction by an angle divided by the largestdetermined focal distance and the focal plane width resolution; and aspheroidal pixel value determiner configured to determine each pixelvalue for each of the spheroidal pixels, according to a pixel value of aplanar pixel projected onto each of the spheroidal pixels.
 6. An imageprocessing apparatus for generating graphics data representing a singleseamless planar image synthesized from a multiple sets of graphics datacontained in a plurality of graphics files, in response to the pluralityof graphics files each of which contains the graphics data composed of amultiplicity of planar pixels arrayed in a plane for representing aplanar image, the image processing apparatus comprising: a synthesisarea establisher configured to establish a cylindrical projection planecentered on a predetermined axis, as an area for synthesis of themultiple sets of graphics data; a cylindrical image generator configuredto generate a plurality of cylindrical images, by projecting each ofplanar images represented by each of the multiple sets of graphics dataonto the projection plane; a feature point extractor configured toextract a feature point which is an area having a predeterminedcharacteristic, from each of the plurality of cylindrical images; acorrespondence relationship determiner configured to determine acorrespondence relationship of the extracted feature points, between theplurality of cylindrical images; a cylindrical image synthesizerconfigured to generate seamless cylindrical graphics data representing asingle seamless cylindrical image, by synthesizing a plurality ofgraphics data each of which representing each of the cylindrical images,with reference to the determined correspondence relationship; and aplanar image generator configured to generate the graphics datarepresenting the single seamless planar image, from the seamlesscylindrical image graphics data.
 7. The image processing apparatus inaccordance with claim 6, wherein the plurality of graphics files furtherinclude image attribute information which is attribute information ofthe graphics data, wherein the image processing apparatus furthercomprises a focal distance determiner configured to determine a focaldistance of an optical system used to generate the multiple sets ofgraphics data for each of the multiple set of graphics data, in responseto the image attribute information; and the axis image generatorgenerates the plurality of axis images by projecting each planar imagerepresented by each of the multiple sets of graphics data onto theprojection plane, the each planar images being placed at a location awayfrom the predetermined axis to the projection plane side, by the focaldistance corresponding to each of the multiple sets of graphics data. 8.The image processing apparatus in accordance with claim 6, wherein thecylindrical image generator establishes the axis parallel to the heightdirection established in the graphics data.
 9. The image processingapparatus in accordance with claim 7, wherein the image attributeinformation includes lens focal distance representing focal distance ofa shooting lens; focal plane resolution unit specifying an unit ofresolution in a focal plane of the optical system; focal plane heightresolution representing a pixel count in a pixel height direction perthe focal plane resolution unit; and focal plane width resolutionrepresenting a pixel count in a pixel width direction per the focalplane resolution unit; the focal distance determiner determines the lensfocal distance to be the focal distance; and the cylindrical imagegenerator determines a pixel size in a width direction by means ofdividing the focal plane resolution unit by focal plane widthresolution, and also determines a pixel size in a height direction bymeans of dividing the focal plane resolution unit by the focal planeheight resolution.
 10. The image processing apparatus in accordance withclaim 7, wherein the image attribute information includes 35mm-equivalent lens focal distance which is a value of focal distanceconverted to a 35 mm film camera basis; the focal distance determinerdetermines the 35 mm-equivalent lens focal distance to be the focaldistance; and the cylindrical image generator determines 35 mm film sizeas a size of the planar image.
 11. The image processing apparatus inaccordance with claim 7, wherein the image attribute informationincludes focal plane resolution unit specifying an unit of resolution ina focal plane of the optical system; focal plane height resolutionrepresenting a pixel count in a pixel height direction per the focalplane resolution unit; and focal plane width resolution representing apixel count in a pixel width direction per the focal plane resolutionunit; the cylindrical image generator comprises: a cylindrical pixelestablisher configured to establish cylindrical pixels on thecylindrical projection plane, the cylindrical pixel being allocated in aheight direction by an angle divided by a largest one of the determinedfocal distances and the focal plane height resolution, and beingallocated in a width direction by an angle divided by the largestdetermined focal distance and the focal plane width resolution; and acylindrical pixel value determiner configured to determine each pixelvalue for each of the cylindrical pixels, according to a pixel value ofa planar pixel projected onto each of the cylindrical pixels.
 12. Animage processing method of generating graphics data representing asingle seamless planar image synthesized from a multiple sets ofgraphics data contained in a plurality of graphics files, in response tothe plurality of graphics files each of which contains the graphics datacomposed of a multiplicity of planar pixels arrayed in a plane forrepresenting a planar image, the image processing method comprising thesteps of: (a) establishing a spheroidal projection plane centered on apredetermined point, as an area for synthesis of the multiple sets ofgraphics data; (b) generating a plurality of spheroidal images, byprojecting each of planar images represented by each of the multiplesets of graphics data onto the projection plane; (c) extracting afeature point which is an area having a predetermined characteristic,from each of the plurality of spheroidal images; (d) determining acorrespondence relationship of the extracted feature points, between theplurality of spheroidal images; (e) generating seamless spheroidalgraphics data representing a single seamless spheroidal image, bysynthesizing a plurality of graphics data each of which representingeach of the spheroidal images, with reference to the determinedcorrespondence relationship; and (f) generating the graphics datarepresenting the single seamless planar image, from the seamlessspheroidal image graphics data.
 13. An image processing method ofgenerating graphics data representing a single seamless planar imagesynthesized from a multiple sets of graphics data contained in aplurality of graphics files, in response to the plurality of graphicsfiles each of which contains the graphics data composed of amultiplicity of planar pixels arrayed in a plane for representing aplanar image, the image processing method comprising the steps of: (a)establishing a cylindrical projection plane centered on a predeterminedaxis, as an area for synthesis of the multiple sets of graphics data;(b) generating a plurality of cylindrical images, by projecting each ofplanar images represented by each of the multiple sets of graphics dataonto the projection plane; (c) extracting a feature point which is anarea having a predetermined characteristic, from each of the pluralityof cylindrical images; (d) determining a correspondence relationship ofthe extracted feature points, between the plurality of cylindricalimages; (e) generating seamless cylindrical graphics data representing asingle seamless cylindrical image, by synthesizing a plurality ofgraphics data each of which representing each of the cylindrical images,with reference to the determined correspondence relationship; and (f)generating the graphics data representing the single seamless planarimage, from the seamless cylindrical image graphics data.
 14. A computerprogram product for causing a computer to generate graphics datarepresenting a single seamless planar image synthesized from a multiplesets of graphics data contained in a plurality of graphics files, inresponse to the plurality of graphics files each of which contains thegraphics data composed of a multiplicity of planar pixels arrayed in aplane for representing a planar image, the computer program productcomprising: a computer readable medium; and a computer program stored onthe computer readable medium, the computer program comprising: a firstprogram for causing the computer to establish a spheroidal projectionplane centered on a predetermined point, as an area for synthesis of themultiple sets of graphics data; a second program for causing thecomputer to generate a plurality of spheroidal images, by projectingeach of planar images represented by each of the multiple sets ofgraphics data onto the projection plane; a third program for causing thecomputer to extract a feature point which is an area having apredetermined characteristic, from each of the plurality of spheroidalimages; a fourth program for causing the computer to determine acorrespondence relationship of the extracted feature points, between theplurality of spheroidal images; a fifth program for causing the computerto generate seamless spheroidal graphics data representing a singleseamless spheroidal image, by synthesizing a plurality of graphics dataeach of which representing each of the spheroidal images, with referenceto the determined correspondence relationship; and a sixth program forcausing the computer to generate the graphics data representing thesingle seamless planar image, from the seamless spheroidal imagegraphics data.
 15. A computer program product for causing a computer togenerate graphics data representing a single seamless planar imagesynthesized from a multiple sets of graphics data contained in aplurality of graphics files, in response to the plurality of graphicsfiles each of which contains the graphics data composed of amultiplicity of planar pixels arrayed in a plane for representing aplanar image, the computer program product comprising: a computerreadable medium; and a computer program stored on the computer readablemedium, the computer program comprising: a first program for causing thecomputer to establish a cylindrical projection plane centered on apredetermined axis, as an area for synthesis of the multiple sets ofgraphics data; a second program for causing the computer to generate aplurality of cylindrical images, by projecting each of planar imagesrepresented by each of the multiple sets of graphics data onto theprojection plane; a third program for causing the computer to extract afeature point which is an area having a predetermined characteristic,from each of the plurality of cylindrical images; a fourth program forcausing the computer to determine a correspondence relationship of theextracted feature points, between the plurality of cylindrical images; afifth program for causing the computer to generate seamless cylindricalgraphics data representing a single seamless cylindrical image, bysynthesizing a plurality of graphics data each of which representingeach of the cylindrical images, with reference to the determinedcorrespondence relationship; and a sixth program for causing thecomputer to generate the graphics data representing the single seamlessplanar image, from the seamless cylindrical image graphics data.